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Executive Summary 


The Technology Immersion Pilot (TIP) sets forth a vision for technology immersion in Texas public 
schools. The Texas Education Agency (TEA) directed nearly $14 million in federal Title II, Paid D 
monies toward funding a wireless learning environment for high-need middle schools through a 
competitive grant process. A concurrent research project funded by a federal Evaluating State 
Educational Technology Programs grant is evaluating whether student achievement improves over 
time as a result of exposure to technology immersion. The Texas Center for Educational Research 
(TCER) — a non-profit research organization in Austin — is the TEA’s primary partner in this landmark 
effort. 

The overarching purpose of the study is to conduct a scientifically based evaluation at the state level to 
test the effectiveness of technology immersion in increasing middle school students’ achievement in 
core academic subjects. Technology immersion encompasses multiple components, including a laptop 
computer for every middle school student and teacher, wireless access throughout the campus, online 
curricular and assessment resources, professional development and ongoing pedagogical support for 
curricular integration of technology resources, and technical support to maintain an immersed campus. 

Technology Immersion 

As a way to ensure consistent interpretation of technology immersion and comparability across sites, 
the TEA issued a Request for Qualifications (RFQ) that allowed commercial vendors to apply to 
become providers of technology immersion packages. Successful vendor applicants to the RFQ had to 
include the following six components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus 
to ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as a learning tool; 

• Online instructional resources that support the state curriculum in English language arts, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, 
learning, and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through a competitive application submission and expert-review process, the TEA selected three lead 
vendors as providers of technology immersion packages (Dell Computer Inc., Apple Computer Inc., 
and Region 1 Education Service Center [ESC]). Prices for packages varied according to the numbers 
of students and teachers, the type of laptop computer, and the vendor provider. Package costs ranged 
from about $1,100 to $1,600 per student. Of the 22 immersion sites, 6 middle schools selected the 
Apple package, 15 selected the Dell package, and 1 school selected the Region 1 ESC package (Dell 
computer). 
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Methodology 


Evaluation Design 

The evaluation employs a quasi-experimental research design with 44 middle schools assigned to 
either treatment or control groups (22 schools in each). Researchers will examine the relationships that 
exist among contextual conditions, technology immersion, intervening factors (school, teacher, and 
student), and student achievement. The research also will determine the impact of immersion on 
student achievement in core subject areas as measured by the Texas Assessment of Knowledge and 
Skills (TAKS). We investigated six research questions in the first year: 

• What are the baseline characteristics of participating schools? 

• How is technology immersion implemented? 

• What is the effect of technology immersion on schools? 

• What is the effect of technology immersion on teachers and teaching? 

• What is the effect of technology immersion on students and learning? and 

• Does technology immersion impact student achievement? 

The Theoretical Framework for Technology Immersion guides the evaluation. The experimental 
research design allows an estimate of the effects of the intervention, which is the difference between 
the treatment and control groups. The framework postulates a linear sequence of causal relationships. 
First, experimental schools arc to be “immersed” in technology through the introduction of technology 
immersion components. Given quality implementation, school-level improvements are expected for 
measures of classroom technology integration, technical support, innovative culture, and parent and 
community support. Leadership and system support drives progress toward full immersion. 

An improved school environment for technology should then lead to teachers who have greater 
technology proficiency, use technology more often for their own professional productivity, collaborate 
more with their peers, have students use technology more and in new ways in their classrooms, and 
use laptops and digital resources to increase the intellectual challenge of lessons. In turn, these 
improved school and classroom conditions should lead students to greater technology proficiency, 
more opportunities for peer collaboration, greater personal self-direction, and stronger engagement in 
school and learning. Student mediating variables presumably contribute to increased academic 
performance as measured by standardized test scores. In the framework, links are also shown between 
student achievement and student, family, and school characteristics, which exert their own influence 
on learning. 

Participating Sites 

Interested districts and associated middle schools responded to a Request for Application (RFA) 
offered by the TEA in spring 2004 to become technology immersion schools. Applicants to become 
TIP sites had to meet eligibility requirements for Title II, Part D funds (i.e., high-need due to children 
from families with incomes below the poverty line, schools identified for improvement, or schools 
with substantial need for technology). Twenty-two technology immersion schools, selected through 
the competitive grant process, were matched by researchers with 22 control schools on key 
characteristics, including size, regional location, demographics, and student achievement. 

The TIP grants targeted high-need schools, thus nearly 70% of students in the study come from 
economically disadvantaged backgrounds, with many schools in rural or isolated locations. Students 
are ethnically diverse, roughly 56% Hispanic and 9% African American. TIP Middle schools are 
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highly concentrated in rural and very small districts across the state. Still, about a third of the districts 
and schools are in large cities or suburban locations in or around cities. The sample also includes 
campus charter schools (one each for the treatment and control group) located in a major urban 
district. 

Three groups or cohorts of students will be followed in the study, with Cohort 1 followed for four 
years, Cohort 2 for three years, and Cohort 3 for two years. In 2004-05, data collection activities 
centered on the initial sixth-grade cohort, which included 5,564 students (2,570 at immersed and 2,994 
at control campuses). About 1,304 teachers participated in the study (622 at immersed and 682 at 
control campuses). 

Data Collection and Analysis 

Data collection involved a mix of qualitative and quantitative data sources. Researchers conducted site 
visits in each of the middle schools in fall 2004 and spring 2005. For this report, we concentrate on 
site- visit data gathered through observations in a sample of sixth-grade classrooms (English/language 
arts, mathematics, social studies, and science). Additional measures, administered as pre- and post- 
measures in fall and spring, include a Campus Technology Inventory completed by the campus 
technology coordinator, teacher online surveys, and student paper-and-pencil surveys. Additionally, 
we gathered school and student demographic, attendance, and achievement data from the Texas Public 
Information Management System (PEIMS) and Academic Excellence Indicator System (AEIS). In 
spring 2005, individual middle schools submitted student-level data on disciplinary actions. 

We analyzed the effects of immersion on teachers’ and students’ self-reported perceptions of 
technology and proficiencies and students’ TAKS achievement using two-level hierarchical linear 
models (HLM). For various analyses contrasting teachers or students in immersed and control schools 
after one school year of implementation, we used important teacher characteristics (fall survey scale 
scores, experience, technology certification, gender) and student characteristics (fall survey scale 
scores, prior achievement, economic and minority characteristics, and gender) as control variables. We 
also calculated effect sizes in standard deviation units (usually Cohen’s d). The interpretation is that an 
effect greater than 0.5 is large, 0.5 to 0.3 is moderate, 0.3-0. 1 is small, and less than 0.1 is trivial. 

Major Findings 

First-year results reveal positive effects of technology immersion on schools (leadership and system 
support, innovative culture, classroom integration, parent and community support), teachers 
(proficiency and productivity, technology use and integration, collaboration), and students (technology 
proficiency and use, small-group work, school satisfaction, and behavior). In most cases, the sizes of 
effects suggest that the impacts of technology immersion are of both statistical and practical 
importance. In contrast to positive effects on school, teacher, and student mediating variables, there 
were no statistically significant effects of immersion in the first year on either reading or mathematics 
achievement for sixth graders, who are members of a student cohort that will be followed through 
eighth grade. Overall, positive findings arc compelling in light of evidence indicating that the level of 
implementation in the first year for 20 of the 22 middle schools was only partial immersion rather than 
substantial (2 schools) or full immersion (no schools). Additional details for key findings are provided 
below. 

First-Year Implementation 

Researchers used rating scales to identify four levels of immersion: minimal (1), partial (2), 
substantial (3), and full (4). The overall level of Technology Immersion was a composite score derived 
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from values for four domains: (a) Robust Access to Technology, (b) Technical and Pedagogical 
Support, (c) Professional Development, and (d) Resource Utilization and related indicators. Scores 
came from various data sources including vendor records, interviews, focus groups, surveys, and grant 
documents. 

In the first year, almost all middle schools achieved only partial immersion. Middle schools 
struggled in the initial year to accommodate the complex demands of technology immersion within the 
existing school environment. As might be expected, no campus reached full immersion. The two 
middle schools that made greater strides toward immersion than others ( substantial immersion) had 
stronger district and campus leadership and invested more time and resources in professional 
development. 

In general, first-year implementation was affected by a number of school and contextual factors. First, 
time for planning was insufficient due to grant-related logistical procedures. Furthermore, many 
middle schools, which were housed in older buildings, encountered problems with outdated 
infrastructures and technical problems with wireless networks and Internet connectivity. Districts and 
campuses also had to grapple with myriad policies and practices related to laptop access and use. The 
greatest barriers to implementation, however, involved people. Teachers were at different stages of 
readiness for immersion and their receptivity varied. Varying abilities and attitudes, coupled with 
teachers’ perceived pressures to improve students’ scores on the TAKS, made many teachers reluctant 
to try new and untested instructional methods and materials in the first year. Additionally, leadership 
at both the district and campus levels emerged as a critical factor driving or limiting progress. 

Effects of Immersion on Schools 

Technology immersion positively affects the school culture, including factors such as innovation, 
collaboration, leadership, parent and community support, and students’ school satisfaction. 

Technology immersion had a statistically significant effect on teachers’ perceptions of four school- 
level factors. Since immersed schools received a wealth of technology resources, it was predictable to 
find that immersed teachers perceived greater availability and use of resources for Classroom 
Technology Integration than control teachers (effect size of 0.56). Teachers in immersed schools also 
reported stronger Leadership and System Support for technology (effect size of 0.20). More 
remarkable, however, was immersed teachers’ perceptions of a more Innovative Culture in their 
middle schools (effect size of 0.35). In particular, teachers at immersed schools were more likely than 
control teachers to share an understanding about the use of technology to enhance student learning, 
and they were less afraid to learn about and try new technologies in their classes. The infusion of 
technology also increased Collaboration among treatment teachers (effect size of 0.41). Teacher 
interactions at immersed schools significantly more often than at control supported improvements in 
instructional practices and exchanges of information about students and their learning. 

The implementation of technology immersion also generated a great deal of excitement in schools and 
communities. This likely contributed to immersed teachers’ belief that their schools have stronger 
Parent and Community Support for technology (effect size of 0.49). Sixth-grade students at immersed 
middle schools also expressed significantly higher levels of School Satisfaction than control students 
(effect size of 0.13). Treatment students were more likely to be satisfied with their school work, 
consider learning more important than the grade received, and see a connection between school work 
and their future life and work. 
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Effects of Immersion on Teachers 


Teachers at immersed schools perceive themselves as more technology proficient than control 
teachers and use technology more productively to support professional practices. In a self- 
assessment of Technology Proficiency in spring, teachers at immersed schools considered themselves 
to be significantly more technology literate than control teachers (effect size of 0.16). Although 
teachers were equally likely to be proficient in technology operations, teachers at immersed schools 
reported greater pedagogical skills in areas such as creating electronic presentations, teaching 
copyright issues, creating technology-integrated lesson plans, and using technology for collegial 
collaboration. Immersed teachers also began to use technology significantly more often than control 
teachers for administrative and classroom management puiposes. Treatment teachers reported greater 
use of technology for Professional Productivity (effect size of 0.37) on indicators such as 
communicating with students, posting information on a website, administering an online assessment, 
and accessing model lesson plans integrating technology. 

Teachers at immersed schools have students use technology more often and they report the use 
of more innovative and learner-centered practices compared to control teachers. With increased 
access to technology, teachers at immersed schools compared to control reported in spring that their 
Students Use Technology significantly more often in their classrooms (effect size of 0.70). For 
example, students more often express themselves in writing (using a word processor), learn and 
practice skills, and conduct Internet research on an assigned topic. Still, treatment teachers’ responses 
suggest that students may do such activities infrequently (i.e., only once or twice a month). Teachers 
at immersed schools also expressed stronger support for Technology Integration (effect size of 0.73). 
For example, they were more likely than control teachers to report that they allocate time for students 
to practice computer skills, plan computer-related activities to improve students’ basic skills, use 
cutting-edge technology, and use computers to promote students’ problem solving and critical 
thinking. Immersed teachers also expressed a stronger affiliation with Learner-Centered Instruction 
(effect size of 0.30). Immersed teachers, for instance, were more likely than control to indicate that 
students establish individual learning goals, engage in experiential learning, and have real-world 
experiences. 

Although teachers at immersed schools use technology more, their lessons typically lack 

intellectual challenge. Technology immersion’s theorized impact on student achievement hinges not 
just on more frequent technology use, but also on technology’s facilitation of more rigorous and 
authentic learning (e.g., high-level thinking, concept formation, inquiry and investigation, access to 
and use of information, exposure to places/resources beyond the classroom, and real-world learning). 
Thus, during fall and spring observations in sixth-grade classrooms, researchers rated the Intellectual 
Challenge of lessons. Rating scales (developed by Newmann, Secada, and Wehlage, 1995) gauged 
Higher Order Thinking, Disciplined Inquiry (Deep Knowledge and Substantive Conversation), and 
Value Beyond School. 

Pre- and post-results for 58 immersed and 57 control teachers revealed no statistically significant 
differences between comparison groups in spring 2005. Nevertheless, fall-to- spring comparisons 
revealed that teachers in immersed classrooms provided slightly more challenging lessons in spring, 
whereas control teachers taught less challenging lessons. More noteworthy, however, was the low 
level of intellectual challenge in class activities for both comparison groups (about 1.6 on the 5-point 
intellectual challenge scale). In many of the observed sixth-grade classrooms, with or without laptop 
use, teachers concentrated on lower order factual knowledge and skills. Lessons frequently involved 
multiple-choice or short-answer worksheets focused on the acquisition of basic skills rather than more 
complex endeavors and higher order thinking. Additionally, lessons often featured brief instructional 
segments across a variety of learning objectives rather than in-depth focus on a topic or concept. 



Moreover, teachers rarely helped students to understand the relevance of their learning or made 
connections with students’ prior experiences. Findings from classroom observations are important 
because of the established link between more challenging and authentic pedagogy and academic 
achievement (Newman & Associates, 1996; Newmann, Bryk, & Nagoaka, 2001). If abundant access 
to technology fails to elevate the quality of students’ learning experiences, the likelihood of a positive 
impact on student achievement may be diminished. 

A major challenge for teachers in the first year was simultaneously learning how to use 
technology and finding time to integrate laptops and digital resources into existing practices. 

Although teachers at immersed schools, as a whole, made substantial progress in the first year, teacher 
proficiency and laptop use varied greatly by teacher, subject area, and school. Decisions about how 
and how often laptops were used for teaching and learning depended on each teacher’s readiness and 
preference. Survey results show that more experienced teachers and male teachers in middle schools 
viewed themselves as less proficient, used technology significantly less often, and expressed lower 
level of support for technology integration. 

Information from classroom observations and field work also suggest that in the initial stages of 
implementation, most teachers maintained their existing pedagogical practices. Teachers typically had 
students use laptops to do the same kinds of activities they previously had completed with paper and 
pencil, such as completing worksheets, typing vocabulary words and definitions, or reviewing for 
multiple-choice tests. This finding is consistent with research showing that teachers progress through 
developmental stages while learning to create technology-infused classroom environments. Many 
teachers at immersed campuses appeared to be at the adoption or adaptation phases, as they were 
using technology to support traditional instruction or integrating new technology into traditional 
classroom practice (Apple Computer Inc., 1995). 

Effects of Immersion on Students 

Students at immersed campuses are more highly engaged in school than control students. 

Increased student engagement is one of the most frequently cited benefits in the research literature for 
one-to-one computing. Likewise, during campus visits, administrators, teachers, and students at 
immersed campuses cited greater student interest and motivation for school and learning as positive 
effects. Other findings corroborate anecdotal perceptions. Surveyed sixth-graders at immersed 
campuses in spring expressed significantly higher levels of satisfaction with their middle schools than 
control students. Additionally, sixth graders at immersed schools were sent to the office for 
disciplinary reasons at a significantly lower rate and had fewer school suspensions than students at 
control schools. Effect sizes for school satisfaction (0.13) and disciplinary measures (0.16 and 0.06), 
however, were small. Also, for another indicator of engagement, the school attendance rate, there was 
no apparent boost for immersed students (effect size of 0.08). 

Technology immersion positively affects sixth graders’ technology proficiency and opportunity 
to use technology. As anticipated, sixth-grade students at immersed middle schools rated their 
Technology Proficiency significantly higher than control students (effect size of 0.47) on items 
measuring the Texas Technology Applications standards. Immersed students felt more capable of 
performing tasks such as sending an email attachment, creating a presentation, managing documents, 
using spreadsheets for graphs, and keeping track of websites. Immersed students’ increased 
proficiency apparently stems from more frequent technology use. Similar to their teachers, surveyed 
sixth graders at immersed schools reported significantly more frequent Technology Use in Core 
Subjects than control students (effect size of 0.96). Flowever, despite large and important increases, 
immersed students’ technology use varied across classrooms and content areas. Treatment students 
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reported using technology most often in reading/English language arts, science, and social studies 
classes (nearly once or twice a week) and least often in math classes (about once or twice a month). 


There was no apparent effect of technology immersion on student self-direction. We theorized 
that sixth graders’ opportunities for independent and self-guided learning afforded through one-to-one 
technology would positively affect students’ personal self-direction. Students completed the Style of 
Learning Inventory as a measure of self-directed learning, including processes such as forethought, 
performance/volition control, and self-reflection. Findings in spring showed there was no significant 
difference between the Self-Directed Learning scale scores for sixth graders in immersed and control 
schools (effect size of 0.06). Nevertheless, changes in students’ perceptions of their self-direction may 
emerge as they progress to higher grade levels and perhaps use their laptops in more and better ways. 

Effects of Immersion on Academic Achievement 

There was no significant effect of technology immersion on sixth graders achievement in reading 
or mathematics. The ultimate goal of technology immersion is increasing middle school students’ 
achievement in core academic subjects as measured by the state assessment (TAKS). In Texas, sixth 
graders complete TAKS assessments for reading and mathematics. We found that after one academic 
year of implementation, there were no positive effects of immersion on either reading or mathematics 
scores. After controlling for prior achievement and other important student characteristics, there were 
no significant differences in the spring 2005 reading or mathematics TAKS z scores of students in 
immersed and control schools. In fact, students in immersed schools had slightly lower scores than 
comparison students. 

Several factors help to explain the discontinuity between the many positive effects noted for schools, 
teachers, and students at immersed campuses and the absence of a positive effect on student 
achievement outcomes. First, implementation fidelity was an important factor. Limited project 
implementation almost certainly influenced outcomes (e.g., the small number of days that students 
actually had laptops, the minimal use of digital resources). In our theoretical model, we hypothesized 
that students in fully immersed schools would experience school and classroom environments that 
would lead to changes in students, which in turn, would lead to increased achievement. While we 
found noteworthy improvements in some areas (e.g., changes in teacher proficiency and technology 
use, improvements in students’ proficiency and school engagement), there were no positive effects on 
students’ personal self-directed learning, and based on classroom observations, the availability of 
laptops did not lead to significantly greater opportunities for students to experience intellectually 
challenging lessons or to do more challenging school work. 

Furthermore, although technology use increased in the first year and surpassed control schools, 
laptops were used infrequently for learning in core subject classes, especially mathematics. Using 
laptops for lessons once or twice a week, or once or twice a month in math classes, may be insufficient 
to make a difference in achievement. Unfortunately, students in Texas middle schools do not complete 
social studies assessment until eighth grade or a science assessment until tenth grade, so we did not 
have academic outcome measures for those content areas. 

It is also important to remember that this is a longitudinal study, and while we expected that some 
impacts might emerge in the first year, it was also considered likely that changes in student academic 
performance would require more than one year to surface. Additionally, the findings reported here 
represent only a first step in analyzing first-year data. Additional analyses will further examine the 
relationships among school, teacher, and student mediating variables and academic achievement. We 
also intend to delve more deeply into the relationships among the fidelity of implementation, 
mediating variables, and outcomes. 
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